Using External Information for More Precise Inferences in General Regression Models

Empirical research usually takes place in a space of available external information, like results from single studies, meta-analyses, official statistics or subjective (expert) knowledge. The available information ranges from simple means and proportions to known relations between a multitude of variables or estimated distributions. In psychological research, external information derived from the named sources may be used to build a theory and derive hypotheses. In addition, techniques do exist that use external information in the estimation process, for example prior distributions in Bayesian statistics. In this paper, we discuss the benefits of adopting generalized method of moments with external moments, as another example for such a technique. Analytical formulas for estimators and their variances in the multiple linear regression case are derived. An R function that implements these formulas is provided in the supplementary material for general applied use. The effects of various practically relevant moments are analyzed and tested in a simulation study. A new approach to robustify the estimators against misspecification of the external moments based on the concept of imprecise probabilities is introduced. Finally, the resulting externally informed model is applied to a dataset to investigate the predictability of the premorbid intelligence quotient based on lexical tasks, leading to a reduction of variances and thus to narrower confidence intervals. Supplementary Information The online version contains supplementary material available at 10.1007/s11336-024-09953-w.


Corollary 1 (Section 3.1)
We start with the proof of Corollary 1: Corollary.Assume θM is the GMM-estimator based on the model estimating equations alone (ignoring the external moments), and that m(z, θ) and θ have the same dimension.Using the prerequisite g(z, θ) = [m(z, θ) T , h(z) T ] T it follows, that Ω has the block form and that (1) Proof.The block form of Ω follows directly.The variance is Var( θex ) = 1 n (G T WG) −1 .Because h(z) does not depend on θ, we have E(∇ θ h(z)) = 0, leading to G = E(∇ θ m(z, θ 0 ) T , 0) T .Using this form of G and partitioning W in the same way as Ω leads to as E(∇ θ m(z, θ 0 )) T is a square matrix and is non-singular because both W M and G T WG are non-singular.Applying results for inverse blocks of partitioned matrices based on Schur complements (Chamberlain, 1987, p. 329, Lemma A.1.) to W and Ω, leads to

Theorem 1 (Section 3.2)
Now we continue with the proof of Theorem 1.
Theorem.Let H = [h(x 1 , y 1 ), . . ., h(x n , y n )] T be the (n×q) random matrix containing the externally informed sample moment functions and 1 n a (n × 1)-vector of ones.Further let Ωh and ΩR be consistent estimators of the corresponding matrices in Corollary 1. Then the (consistent) externally informed OLS estimator and its variance is where σ 2 is the variance of the error in the assumed linear model.
The variance of the estimator shown in Theorem 1 can be seen as a special case of the variance formula in Corollary 1 and it was also derived by Hellerstein and Imbens (1999), hence we will only derive βex here: Proof.Using the notation of Definition 2, the regularity conditions are fulfilled for the externally informed linear model.The first order conditions for the GMM-estimator are ĜT Ŵ[ 1 n n i=1 g(z i , θ)] = 0 (Newey & McFadden, 1994)[p. 2145], where Ĝ is a consistent estimator for G.In the multiple linear case Ω−1 in the same manner as Ω and solving for β we get The second order derivative is −X T X ŴM X T X, which is negative definite if X has full column rank, which proves that βex is indeed the searched maximum according to Definition 2. The structure of Ŵ as a partitioned inverse provides the equality Ŵ−1 This completes the proof.

Expressions in Table 1 (Section 3.2)
We continue with the proof for the expressions in Table 1: Note: The subscript ex indicates externally determined values.In the last line, β xj ,y represents the expected value of the estimator of the slope from a simple linear regression model, which is identical to the true value of the slope only if x j is independent of the other explanatory variables.
Proof.We only have to prove the correctness of the third column (the one for We can omit the exact values of the external moments, as they are constants and as has the expected value 0. For the first row we get by the Gauss-Markov-assumptions.The second row follows by the same argument just with the additional factor x j .For the second moment of y it follows that If the errors are assumed to be at least symmetrically distributed, the first summand vanishes, leaving the term written in the third row in Table 1.For the next row, we rewrite (y − E(y)) 2 as y 2 − 2yE(y) + E(y) 2 and use the linearity of the expected value.Then the Ω T R of the fourth row is just the one in the fourth row minus 2E(y) times the one in the second row.This is 2σ which is written in the fourth row.The expression in the fifth row is derived in the same manner as we can write The expected value of the second and the fourth term is zero, while the first term is equal to Ω T R for the moment E(x j y) and the third term is equal to ), which is the vector of the covariances written in the fifth row.The last two rows follow from the fifth row, treating σ x j and σ y as constants.

Expressions in Table 2 (Section 3.2)
Effects of various single moments in terms of variance reduction.
moments D effect on The expression e j denotes the (p × 1)-vector with 1 at the j-th position and zeros elsewhere.Further we set ẽj := −E(x j ) • e 1 + e j .In the last line, β xj ,y represents the expected value of the estimator of the slope from a simple linear regression model, which is identical to the true value of the slope only if x j is independent of the other explanatory variables.
Proof.To prove the results in Table 2 it is sufficient to use Theorem 1.As ω h is single valued, it holds that )e j using the notation of Table 2 and noting that x 1 = 1, we get the results for [E(xx T )] −1 Ω T R in Table 6.
This proves the results in Table 2.
To illustrate how to determine ω h , the case E(y 2 ) is treated.Using ∼ N (0, σ 2 ) and the Gauss-Markov-assumptions, we get 3 Detailed results of the simulations (Section 5) 3.1 Correctly specified external moments (Section 5.2.1)

Misspecified external moments (5.2.2)
Note.The expressions βex , Var( βex ), Var( βex ),|CI| and | CI| are defined in the beginning of Section 5.2.The results for the moment E(x 2 ) are equivalent to the OLS results.Cov is the coverage for the external point value and Cov I symbolizes the coverage for the confidence interval union based on the external interval.Only the affected coefficients are reported per moment.The true values are β 1 = 1, β 2 = 0.5 and β 3 = 2.
Note.Note: The third and fourth columns contain the recomputed results of in terms of Pluck & Ruales-Chieruzzi (2021) the OLS regression coefficients βj , where β1 is the intercept and β2 is the slope and the robust standard errors s( βj ) of the coefficients.The (robust) 95% confidence intervals CI 0.95 for the parameters were computed in addition.The estimator interval [ βj , βj ], the standard error interval [s( βj ), s( βj )] and the 95% confidence interval union CI 0.95 are shown as results of the estimation of the externally informed model.

Table 7 :
Results of the simulations with correctly specified external moments for sample size n = 30.The expressions βex , Var( βex ), Var( βex ), ∆j ,|CI| and | CI| are defined in the beginning of Section 5.2.The results for the moment E(x 2 ) are equivalent to the OLS results.Cov is the coverage for the external point value and Cov I symbolizes the coverage for the confidence interval union based on the external interval.Only the affected coefficients are reported per moment.The true values are β 1 = 1, β 2 = 0.5 and β 3 = 2.

Table 8 :
Results of the simulations with correctly specified external moments for sample size n = 50.
Note.The expressions βex , Var( βex ), Var( βex ), ∆j ,|CI| and | CI| are defined in the beginning of Section 5.2.The results for the moment E(x 2 ) are equivalent to the OLS results.Cov is the coverage for the external point value and Cov I symbolizes the coverage for the confidence interval union based on the external interval.Only the affected coefficients are reported per moment.The true values are β 1 = 1, β 2 = 0.5 and β 3 = 2.

Table 9 :
Results of the simulations with correctly specified external moments for sample size n = 100.CI| are defined in the beginning of Section 5.2.The results for the moment E(x 2 ) are equivalent to the OLS results.Cov is the coverage for the external point value and Cov I symbolizes the coverage for the confidence interval union based on the external interval.Only the affected coefficients are reported per moment.The true values are β 1 = 1, β 2 = 0.5 and β 3 = 2.

Table 10 :
Results of the simulations with misspecified external moments for sample size n = 15.CI| are defined in the beginning of Section 5.2.The results for the moment E(x 2 ) are equivalent to the OLS results.Cov is the coverage for the external point value and Cov I symbolizes the coverage for the confidence interval union based on the external interval.Only the affected coefficients are reported per moment.The true values are β 1 = 1, β 2 = 0.5 and β 3 = 2.

Table 11 :
Results of the simulations with misspecified external moments for sample size n = 30.CI| are defined in the beginning of Section 5.2.The results for the moment E(x 2 ) are equivalent to the OLS results.Cov is the coverage for the external point value and Cov I symbolizes the coverage for the confidence interval union based on the external interval.Only the affected coefficients are reported per moment.The true values are β 1 = 1, β 2 = 0.5 and β 3 = 2.

Table 12 :
Results of the simulations with misspecified external moments for sample size n = 100.